home *** CD-ROM | disk | FTP | other *** search
- Newsgroups: comp.sources.misc
- subject: v08i047: secure comp.mail.maps unpacker
- from: allbery@uunet.UU.NET (Brandon S. Allbery - comp.sources.misc)
- Reply-To: clewis@eci386 (Chris Lewis)
-
- Posting-number: Volume 8, Issue 47
- Submitted-by: clewis@eci386 (Chris Lewis)
- Archive-name: unpkmaps2
-
- Periodically, people ask for a map unpacker. I've posted this
- before in news.admin, but I guess it's time to make it semi-official.
- The one big advantage to this one is that it's intended to be secure,
- and is in fact somewhat better than uuhosts (not to mention a little
- less of a CPU and disk hog). It's been in use for about 8 months
- on a couple of machines.
-
- I call it "unpackmaps V2.0"
-
- #! /bin/sh
- # This is a shell archive. Remove anything before this line, then unpack
- # it by saving it into a file and typing "sh file". To overwrite existing
- # files, type "sh file -c". You can also feed this as standard input via
- # unshar, or by typing "sh <file", e.g.. If this archive is complete, you
- # will see the following message at the end:
- # "End of archive 1 (of 1)."
- # Contents: MANIFEST README path.patch unpackmaps
- # Wrapped by clewis@eci386 on Tue Sep 19 15:27:00 1989
- PATH=/bin:/usr/bin:/usr/ucb ; export PATH
- if test -f 'MANIFEST' -a "${1}" != "-c" ; then
- echo shar: Will not clobber existing file \"'MANIFEST'\"
- else
- echo shar: Extracting \"'MANIFEST'\" \(275 characters\)
- sed "s/^X//" >'MANIFEST' <<'END_OF_FILE'
- X File Name Archive # Description
- X-----------------------------------------------------------
- X MANIFEST 1 This shipping list
- X README 1 Readme
- X path.patch 1 patch for pathalias
- X unpackmaps 1 map unpacker
- END_OF_FILE
- if test 275 -ne `wc -c <'MANIFEST'`; then
- echo shar: \"'MANIFEST'\" unpacked with wrong size!
- fi
- # end of 'MANIFEST'
- fi
- if test -f 'README' -a "${1}" != "-c" ; then
- echo shar: Will not clobber existing file \"'README'\"
- else
- echo shar: Extracting \"'README'\" \(6341 characters\)
- sed "s/^X//" >'README' <<'END_OF_FILE'
- X UNPACKMAPS V2.0
- X
- XThis is source for a simple, *secure*, map unpacking facility.
- X
- XIt is much simpler than uuhosts, though it will grow somewhat.
- XThe intent is that any UNIX/XENIX/BSD system that can run news will
- Xbe able to run this too, so I'm attempting to keep to greatest-common-
- Xdenominator except for pathalias and things that I implement
- Xmyself.
- X
- XThis release includes a patch to pathalias 9 to allow pathalias to
- Xhandle compressed files in its argument list. You don't need to
- Xpatch pathalias, but if you don't, you can't keep the maps in compressed
- Xformat (yet).
- X
- XThe next release (if there ever is one) will contain such things as
- Xmore efficient path file modification, a method for viewing the map
- Xfiles analogously to uuhosts (from which this software derives a
- Xconsiderable amount of inspiration), and more automated installation.
- X
- XA slightly older version of this was mailed to Neil Gorsuch (of Zardoz
- Xsecurity mailing list fame) who said that he would be working on it,
- Xpossibly merging it with other map unpackers and officially posting it.
- XI'm posting it because I occasionally see requests for it, and Neil
- Xhasn't posted his yet.
- X
- XRegarding security: as many may remember, there's been a fair bit of
- Xdiscussion on security of map unpacking on the net. Rather than play
- Xaround with trying to make a secure *true* unshar, which probably noone would
- Xtrust because it would be so big, I simply made a few simplifying assumptions
- Xabout the map format and use an awk script to unpack a map article into
- Xa map file. It checks for and refuses to unpack articles which have
- Xslashes in their names. I sent off some mail to Mel asking whether the
- Xassumptions I've made about map format are true, but never got any
- Xresponse. I believe that this is *pretty* secure, in that it doesn't
- Xhave to be run as root, doesn't use the Bourne shell for unpacking, and
- Xis careful about the file names it creates. Please let me know if there
- Xare any holes I didn't think of.
- X
- XGeneral operation:
- X - your news is modified to batch incoming map article file names
- X to a specific batch file (analogous to normal news batching).
- X C-news users take note: you may have to utter magic incantations
- X (hint "classes" in C-news Alpha) to get sendbatches to avoid
- X trying to uux these...
- X - unpackmaps wakes up, usually once per day, and extracts the
- X maps specified (if any) in the batch file into the map directory.
- X Maps are extracted using a secure awk script without resort
- X to setuid root or other wierdnesses.
- X - If any maps were extracted, pathalias is fired off, and the
- X resultant file put in the place specified.
- X - if anything was done, unpackmaps sends you mail telling you
- X what happened.
- X
- XInstallation:
- X - If you want to compress the stored map files, patch your
- X version 9 pathalias source using the path.patch file. DO NOT
- X define COMPRESS in unpackmaps unless you've done so.
- X All this patch does is allow pathalias to handle files arguments
- X that may be compressed - it does it by "system"'ing off COMPRESS when
- X files with ".Z" are found. (I wanted to "popen" COMPRESS, but
- X the changes looked a little extreme, besides, having both
- X compress and pathalias running on a pipeline seemed to me to
- X be a little extreme w.r.t. memory.) You should, after patching
- X pathalias, modify the COMPRESS define in pathalias's config.h
- X to contain the same pathname as the COMPRESS in unpackmaps,
- X except that the COMPRESS define in config.h should also include
- X "-d" (it's being used as if it was zcat)
- X
- X uuhosts does compression by completely uncompressing the map
- X database, and running pathalias, and then recompressing the whole
- X thing again. Which means you need an enormous amount of disk
- X (approx 3 times the compressed map directory), plus be willing
- X to put up with a compression run on every map file too. (not
- X to mention intermediate path files)
- X
- X My approach only needs enough space for one additional uncompressed
- X map entry plus a copy or two of the final map database.
- X
- X - I STRONGLY recommend that you build a special version of compress
- X with 12 bit compression instead of 16 for this. When this
- X is done, compress is considerably smaller (eg: bss of 32K instead
- X of 400K+). The reason for this is obvious - pathalias is enormous
- X when it's running, and it ain't nice to have to fork/exec
- X a 500K+ process at the same time for *every* map file. Ugh.
- X
- X Advantages:
- X - the whole thing runs considerably faster
- X - much less swap/paging
- X - on our machine, 16 bit compress practically hangs
- X everyone else when run at the same time as pathalias.
- X Disadvantages:
- X - the map directory is 10% (really! only 10%!) bigger.
- X
- X What I did was the following:
- X - go to the source directory for compress
- X - remove the binary if it is there.
- X - say:
- X make compress CFLAGS=-DUSERMEM=0
- X - rename this to something like /usr/bin/compress12
- X - make sure that the config.h for pathalias and unpackmaps
- X shell script has the same name. Eg: /usr/bin/compress12
- X
- X - edit unpackmaps to set the variables at the beginning of
- X the shell script.
- X - make the directory for the map files, owned by news.
- X - put unpackmaps in a suitable place. Eg: /usr/lib/news
- X - unpackmaps should be run from the userid that owns and runs news.
- X - su to the news userid, and run:
- X unpackmaps -i
- X This will build the initial path file.
- X - insert into your crontab something like:
- X 30 3 * * * /bin/su news -c "<path to unpackmaps>/unpackmaps > /dev/null"
- X
- Xunpackmaps -i: will extract all map articles into the map spool area - useful
- Xfor the first time you use it.
- X
- Xunpackmaps -p: runs pathalias even if no map articles were extracted.
- X
- XIn order to push the paths file into /usr/lib/uucp, I created a file called
- Xpaths in /usr/lib/uucp, with 644 permissions, owned by the userid that runs
- Xunpackmaps.
- X
- XThe package will send mail to who you specify indicating which maps were
- Xunpacked, and any error returns from pathalias.
- X
- XLet me know of any changes you needed to make to get this to work.
- XI'm also open to suggestions for new features....
- X----------
- XChris Lewis, Markham, Ontario, Canada
- X{uunet!attcan,utgpu,yunexus,utzoo}!lsuc!{ecicrl|eci386}!clewis
- XFerret Mailing list: ...!lsuc!gate!eci386!ferret-request
- X(or lsuc!gate!eci386!clewis or lsuc!clewis)
- END_OF_FILE
- if test 6341 -ne `wc -c <'README'`; then
- echo shar: \"'README'\" unpacked with wrong size!
- fi
- # end of 'README'
- fi
- if test -f 'path.patch' -a "${1}" != "-c" ; then
- echo shar: Will not clobber existing file \"'path.patch'\"
- else
- echo shar: Extracting \"'path.patch'\" \(2225 characters\)
- sed "s/^X//" >'path.patch' <<'END_OF_FILE'
- X*** config.old.h
- X--- config.h
- X**************
- X*** 11,16
- X /* default place for dbm output of makedb (or use -o at run-time) */
- X #define ALIASDB "/usr/local/lib/palias"
- X
- X
- X
- X /**************************************************************************
- X--- 11,19 -----
- X /* default place for dbm output of makedb (or use -o at run-time) */
- X #define ALIASDB "/usr/local/lib/palias"
- X
- X+ /* place for zcat for uncompressing maps inline */
- X+ #define COMPRESS "/usr2/clewis/maps/compress12 -d"
- X+
- X
- X
- X /**************************************************************************
- X*** parse.old.y
- X--- parse.y
- X**************
- X*** 402,407
- X fixprivate(); /* munge private host definitions */
- X Lineno = 1;
- X while (optind < Argc) {
- X if (freopen((Cfile = Argv[optind++]), "r", stdin) != 0)
- X return 0;
- X sprintf(errbuf, "%s: %s", Argv[0], Cfile);
- X--- 402,439 -----
- X fixprivate(); /* munge private host definitions */
- X Lineno = 1;
- X while (optind < Argc) {
- X+ #ifdef COMPRESS
- X+ static char Fname[512];
- X+ char compressed[512];
- X+ char cmdbuf[512];
- X+ int dotidx;
- X+
- X+ strcpy(Fname, Cfile = Argv[optind++]);
- X+ strcpy(compressed, Cfile);
- X+
- X+ dotidx = strlen(Cfile) - 2;
- X+ if (dotidx >= 0 && strcmp(&Fname[dotidx], ".Z") == 0)
- X+ Fname[dotidx] = '\0';
- X+ else
- X+ strcat(compressed, ".Z");
- X+
- X+ if (access(Fname, 04) != 0) {
- X+ if (access(compressed, 04) == 0) {
- X+ unlink("/tmp/pathfile");
- X+ sprintf(cmdbuf, "%s < %s > /tmp/pathfile", COMPRESS,
- X+ compressed);
- X+ system(cmdbuf);
- X+ if (freopen("/tmp/pathfile", "r", stdin) != 0) {
- X+ unlink("/tmp/pathfile");
- X+ return 0;
- X+ }
- X+ unlink("/tmp/pathfile");
- X+ }
- X+ } else {
- X+ if (freopen(Fname, "r", stdin) != 0)
- X+ return 0;
- X+ }
- X+ #else
- X if (freopen((Cfile = Argv[optind++]), "r", stdin) != 0)
- X return 0;
- X #endif
- X**************
- X*** 404,409
- X while (optind < Argc) {
- X if (freopen((Cfile = Argv[optind++]), "r", stdin) != 0)
- X return 0;
- X sprintf(errbuf, "%s: %s", Argv[0], Cfile);
- X perror(errbuf);
- X }
- X--- 436,442 -----
- X #else
- X if (freopen((Cfile = Argv[optind++]), "r", stdin) != 0)
- X return 0;
- X+ #endif
- X sprintf(errbuf, "%s: %s", Argv[0], Cfile);
- X perror(errbuf);
- X }
- END_OF_FILE
- if test 2225 -ne `wc -c <'path.patch'`; then
- echo shar: \"'path.patch'\" unpacked with wrong size!
- fi
- # end of 'path.patch'
- fi
- if test -f 'unpackmaps' -a "${1}" != "-c" ; then
- echo shar: Will not clobber existing file \"'unpackmaps'\"
- else
- echo shar: Extracting \"'unpackmaps'\" \(5874 characters\)
- sed "s/^X//" >'unpackmaps' <<'END_OF_FILE'
- X# Written by: Chris Lewis, clewis@eci386
- X# unpackmaps release 2.0 (alpha)
- Xtrap "rm -f /tmp/unp?$$; exit" 0 1 2 3 15
- XIFS="
- X"
- Xexport IFS
- XPATH=/bin:/usr/bin
- Xexport PATH
- X
- X# The name of the file that you've caused your news system to
- X# batch the file names of the map articles.
- X# Eg: C-news
- XBATCH=/usr/lib/news/batch/b.maps/togo
- X# Eg: B-news
- X#BATCH=/usr/spool/batch/maps
- X# News spool directory
- XNEWSSPOOL=/usr/spool/news
- X# Where you want the maps to go.
- X# I like using /usr/spool/maps, but on our system, /usr/spool/news
- X# is a separate file system, and /usr runs close to the limit...
- XMAPDIR=/usr/spool/news/maps
- X# Person to send results and error messages to
- XNOTIFY=clewis
- X# pathalias binary
- XPATHALIAS=/usr/lbin/pathalias
- X# where you want the path files to go:
- X# A convenient place is /usr/lib/uucp/paths which is the smail
- X# default. If you're going to put this in /usr/lib/uucp, I suggest
- X# (rather than make /usr/lib/uucp writeable by everybody), doing
- X# the following:
- X# su root
- X# cd /usr/lib/uucp
- X# touch paths
- X# chown news paths (or usenet)
- X# chmod 644 paths
- XPATHFILE=/usr/lib/uucp/paths
- X# Auxiliary options to pathalias. Tune to local tastes....
- XPATHOPTS="-dwatmath"
- X# If you have a version[s] of your machine's map entry that is different
- X# from what's published, change this variable to point at it/them.
- X# (Eg: I publish the first entry here, and the second one is local tuning
- X# and hidden connections)
- XPATHLOCAL="/usr2/clewis/maps/path.local /usr2/clewis/maps/path.nonpublic"
- X# If this variable is set to the compress binary, maps will be
- X# compressed. DO NOT define this unless you've applied the accompanying
- X# patch to your version 9 pathalias source, and you've set the path
- X# in config.h to point at the same place.
- XCOMPRESS=/usr2/clewis/maps/compress12
- X# 1 to strip comments from maps - don't do this if you ever try
- X# to read the maps and figure out where the site is, their contacts
- X# etc.
- XNOCOMMENTS=0
- X# PS: there is *one* possible edit that you might want to make
- X# below - the maps used to generate wierd domains, but most of that
- X# appears to be gone now (don't ask me, I never particularly understood
- X# it, but since Peter Honeyman recommended it...). If you object
- X# to these wierd domains, uncomment the egrep.
- X
- X# Edit no more....
- X
- Xumask 022
- X
- XPATH=/bin:/usr/bin
- Xexport PATH
- X
- Xif test ! -d $MAPDIR -o ! -w $MAPDIR
- Xthen
- X echo "$MAPDIR missing, unwritable or not a directory" >&2
- X exit
- Xfi
- X
- Xif [ $# = 1 ]
- Xthen
- X case $1 in
- X -p)
- X forcepath=true
- X ;;
- X -i)
- X cd /
- X rm -f $BATCH.work
- X # using find/sort instead of ls just in case there's lots of
- X # articles....
- X find $NEWSSPOOL/comp/mail/maps -type f -print | sort > $BATCH
- X ;;
- X *)
- X echo "usage: unpackmaps [-i] [-p]" >&2
- X exit 1
- X ;;
- X esac
- Xfi
- X
- Xcd $MAPDIR
- X
- Xwhile [ -f $BATCH -o -f $BATCH.work ]
- Xdo
- X # There is no window of vulnerability here as long as noone else is
- X # creating $BATCH.work.
- X if [ ! -f $BATCH.work ]
- X then
- X mv $BATCH $BATCH.work
- X fi
- X
- X while read i stuff
- X do
- X # Using stuff to capture remaining junk on line.
- X # Eg: C-news article sizes.
- X
- X if [ -z "$i" ]
- X then
- X break
- X fi
- X
- X if [ ! -r $i ]
- X then
- X echo "$i apparently superseded or expired"
- X continue
- X fi
- X
- X # This awk script depends on the following map article format:
- X # <don't cares>
- X # cat << 'something' > filename
- X # map body
- X # something
- X # <don't cares>
- X # "something" doesn't have to be enclosed in quotes in the cat line.
- X # This isn't particularly fast - could be dramatically speeded up
- X # if written in C, but I was trying to ensure that this is as simple
- X # and self-evident as possible.
- X
- X awk '
- X $1 == "cat" {
- X endtoken=$3;
- X if (substr(endtoken, 1, 1) == "'"'"'")
- X endtoken=substr(endtoken, 2, length(endtoken)-2);
- X collecting = 1;
- X foundone = 1;
- X name = $5;
- X if (index(name, "/") != 0) {
- X printf("Security violation attempt in %s!\n", "'$i'");
- X exit;
- X } else
- X printf("extracting %s from %s\n", name, "'$i'");
- X next;
- X }
- X
- X {
- X if (!collecting)
- X next;
- X if ($1 == endtoken) {
- X line = "rm -f " name ".Z"
- X print "" | line
- X collecting = 0;
- X next
- X }
- X if ("'$NOCOMMENTS'" == 1 && $0 ~ /#/)
- X print substr($0, 1, index($0, "#")) > name
- X else
- X print $0 > name
- X }
- X
- X END {
- X if (collecting) {
- X printf("Non-terminated map in %s\n", "'$i'");
- X }
- X if (!foundone) {
- X printf("%s does not contain a properly formed map\n", "'$i'");
- X }
- X }' $i
- X done < $BATCH.work
- X rm $BATCH.work
- Xdone > /tmp/unpA$$ 2>&1
- X
- Xif test -n "$PATHALIAS" -a -x "$PATHALIAS"
- Xthen
- X if test -s /tmp/unpA$$ -o -n "$forcepath"
- X then
- X cd $MAPDIR
- X
- X (
- X $PATHALIAS -f $PATHOPTS ?.* $PATHLOCAL |
- X
- X # format of the pathalias -f output is
- X # cost host route
- X #
- X # format of a 'paths' file for smail is
- X # host route first_hop_cost
- X #
- X # move cost field to end of line:
- X
- X sed 's/\(.*\) \(.*\) \(.*\)/\2 \3 \1/' |
- X
- X # convert target domain/host to lower case:
- X
- X #lcasep |
- X
- X # remove some additional wierdnesses (per Peter Honeyman):
- X # You can leave it in or not.
- X
- X # egrep -v '(\.(com|edu|mil|gov|net|org|arpa|[a-z][a-z]) .*!.*!)|(.\.(com|edu|mil|gov|net|org|arpa|[a-z][a-z]) )' |
- X
- X # sort the stream:
- X
- X sort > /tmp/paths ) > /tmp/unpB$$ 2>&1
- X
- X if test ! -s /tmp/paths
- X then
- X echo "Pathalias failed no map file created" >> /tmp/unpB$$
- X else
- X cat /tmp/paths > $PATHFILE 2>> /tmp/unpB$$
- X if test $? != 0
- X then
- X echo "Copy to $PATHFILE failed" >> /tmp/unpB$$
- X else
- X rm /tmp/paths
- X fi
- X echo "Map remade" >> /tmp/unpB$$
- X fi
- X
- X if test -s /tmp/unpB$$
- X then
- X echo "Pathalias output:" >> /tmp/unpA$$
- X cat /tmp/unpB$$ >> /tmp/unpA$$
- X fi
- X fi
- Xfi
- X
- Xif test -x "$COMPRESS"
- Xthen
- X files=`ls ?.* | sed -e '/\.Z$/d'`
- X if [ -n "$files" ]
- X then
- X $COMPRESS -f $files
- X fi
- Xfi
- X
- X
- Xif test -s /tmp/unpA$$
- Xthen
- X mail $NOTIFY < /tmp/unpA$$
- Xfi
- END_OF_FILE
- if test 5874 -ne `wc -c <'unpackmaps'`; then
- echo shar: \"'unpackmaps'\" unpacked with wrong size!
- fi
- chmod +x 'unpackmaps'
- # end of 'unpackmaps'
- fi
- echo shar: End of archive 1 \(of 1\).
- cp /dev/null ark1isdone
- MISSING=""
- for I in 1 ; do
- if test ! -f ark${I}isdone ; then
- MISSING="${MISSING} ${I}"
- fi
- done
- if test "${MISSING}" = "" ; then
- echo You have the archive.
- echo "Read README to install unpackmaps"
- rm -f ark[1-9]isdone
- else
- echo You still need to unpack the following archives:
- echo " " ${MISSING}
- fi
- ## End of shell archive.
- exit 0
-
- --
- Chris Lewis, R.H. Lathwell & Associates: Elegant Communications Inc.
- UUCP: {uunet!mnetor, utcsri!utzoo}!lsuc!eci386!clewis
- Phone: (416)-595-5425
-
-
-